Rhythm and Time in Music Epitomize the Temporal Dynamics of Human Communicative Behavior: The Broad Implications of London’s Trinity

Three key issues about rhythm and timing in music are drawn to the attention of linguists in a paper by London (2012). In this commentary, I argue that these issues are relevant not only to linguists, but also to those in any field dealing with the temporal dynamics of human communicative behavior. Thus, the distinction between endogenously and exogenously driven mechanisms of perceptual organization, the active nature of perception, and the presence of multiple time scales are topics that also concern experimental psychologists and cognitive neuroscientists. London’s argument that these three issues play a crucial role in the perception of rhythm and timing implies that they should be considered collectively when attempting to understand diverse communicative acts. Submitted 2012 January 6; accepted 2012 July 13. KEYWORDS: rhythm perception, timing, brain, embodiment, communication IN his cogent paper, music theorist Justin London identifies three issues about rhythm and time in music that, he suggests, may be useful for linguists to consider. These issues—which are often overlooked or misrepresented even in the music domain—pertain to (1) the distinction between meter and grouping, (2) the active nature of perception, and (3) the simultaneous presence of multiple time scales. The present commentary discusses how these three topics relate more generally to the temporal dynamics of human communicative behavior. Taking a tutorial approach, I will draw on work from two interconnected and broad fields: experimental psychology, which endeavors to understand the mental processes that have evolved to allow humans to interact with their environments, and cognitive neuroscience, which endeavors to uncover the neural substrates of these mental processes. It turns out that each of London’s “three things linguists need to know” may have implications not only for linguists, but also for experimental psychologists and cognitive neuroscientists who are interested in understanding the communication of meaning and aesthetic experience in verbal and nonverbal domains. THE DUALITY OF METER AND GROUPING The distinction that London (2012) draws between metric and grouping structure in musical rhythm represents a duality. Meter is the endogenous sense of hierarchically arranged levels of pulsation that give rise to the experience of cyclic patterns of beats. One of the levels, the ‘tactus’, is (typically) especially salient, and has a tendency to induce periodic body movement. Grouping involves the perceptual organization of intervals defined by sound onsets into patterns that may vary in the complexity of ratio relations between their constituent intervals. Group structure is determined by accents associated with changes in pitch, intensity, duration, and/or timbre. The essence of the distinction is that meter is not physically in the music, while cues to grouping are. London (2012) provides a compendium of similarities and differences in the essential characteristics of meter and grouping in his Table 1. The implications of this list are threefold. First, meter and grouping may support different, albeit complementary functions in music perception and production. Second, meter and grouping are based on different underlying processes (endogenous beats vs. the perception of accents stemming from physical properties of sounds). Third, meter and grouping are subject to different constraints (which are much stricter for meter). For example, the measures of meter are cyclical and contiguous while groups are not necessarily so. Differing constraints mean that metric and grouping structure may not always align. London (2012) illustrates this through the use of a melody with an anacrusis (i.e., the melody begins on a weak metric location: an upbeat). Thus, while Empirical Musicology Review Vol. 7, No. 1-2, 2012 18 the same sonic event can simultaneously mark a metric and a group boundary, this is not necessarily the case. Just as dualities—of mind and body, particle and wave, etc.—have traditionally tended to vex even the most brilliant minds, the distinction between meter and grouping has proven to be hard to grasp. In a sense, the difficulty is reflected in dichotomous approaches to understanding how the mechanisms that underlie meter and grouping are instantiated in the nervous system. On the one hand, there are dynamic systems approaches that model rhythmic behavior using nonlinearly coupled oscillators (see Large, 2008), while, on the other hand, there are information processing based approaches that favor linear autoregressive models and memory representations of discrete time intervals (e.g., Vorberg & Wing, 1996). The dynamic and information-processing based approaches, when taken together as a package, seem tailor-made for dealing with phenomena related to meter and grouping. On one hand, the periodic, self-sustained nature of the oscillations that pervade dynamic systems formulations are clearly well suited to instantiate cyclical metric pulsations. The validity of these approaches has been boosted by recently discovered evidence for metric hierarchies in brain activity, found using electroencephalography (EEG), where electrical signals reflecting neural activity are recorded from the scalp (Nozaradan, Peretz, Missal, & Mouraux, 2011; Zanto, Snyder, & Large, 2006), and magnetoencephalography (MEG), where related changes in magnetic fields are recorded (Iversen, Repp, & Patel, 2009). An attractive hypothesis is that these metric oscillations in neural activity reflect fluctuations in attentional energy that are optimized for processing musical rhythm (Jones, 2009; Large, 2008). On the other hand, the relatively liberal nature of interval-based approaches (e.g., Vorberg & Hambuch, 1978), which deal with discrete durations as well as periodic intervals, seems well suited to forming groups of variable size and complexity. Another advantage of these approaches is their long history. Much has been learnt about the identity of the brain regions that underpin interval timing through the application of invasive methods (including single cell recording, surgical lesions, neuropharmacological intervention, and genetic modification) in research with nonhuman animals (Buhusi & Meck, 2005; Coull, Cheng, & Meck, 2011). Dynamic systems and information-processing based approaches to rhythm perception and production have, until recently, proceeded largely independently. Efforts to reconcile them range from formal unification (Pressing, 1999; Schöner, 2002) to views that they are complementary (e.g., Torre & Balasubramanian, 2009). In accordance with the latter perspective, it may be the case that the different computational processes described by dynamical and information processing approaches serve different functions. London (2012) flags a relevant issue in his claim that meter is “strongly predictive”—that is, it is informative about when events will occur—while grouping is “weakly predictive”—it is informative about what will occur. Thus, meter and grouping both relate to expectancies, albeit of different varieties. Expectancies are central to basic perception and aesthetic responses to music (Huron, 2006). In a general sense, expectancy is the experience of perceptually anticipating an event. Such anticipation serves to enhance the processing of the event by preparing—or ‘priming’ by selectively mobilizing appropriate resources—the perceiver for the event’s occurrence (Jones, 1976). There are at least two varieties of expectancies (see Bharucha, 1987; Stevens & Byron, 2009), which may be congruent or incongruent with one another. Schematic expectancies rely on knowledge about statistical regularities that are learnt during the course of years of experience. They are deep, general, and apply to music of a particular culture. Veridical expectancies refer to actual events in a specific musical piece and are tied to the surface structure. Meter can be viewed as a framework for schematic expectancies, whereas rhythmic grouping is more a matter of veridical expectancies (although grouping may also be influenced by schematic expectancies, such as those reflected in Gestalt principles of organization; see Bregman, 1990). Linking meter and grouping to expectancies implies that they may have implications for the aesthetic responses, as these responses are believed to be modulated by the degree to which sequential events confirm or violate a listener’s expectations (Huron, 2006; Meyer, 1956). In any case, attaching meter to schematic expectancies and grouping to veridical expectancies may serve to make the above-mentioned duality concrete. The duality can be further fleshed out by tying meter and grouping to different neurophysiological processes in the brain. Accordingly, studies employing techniques such as functional Magnetic Resonance Imaging (fMRI), which uses changes in cerebral blood oxygenation levels as an index of neural activity, have revealed different brain circuits underlying meter and grouping. The experience of meter may be driven by timing-keeping circuits spanning brain regions that include premotor cortex, supplementary motor area, basal ganglia, and cerebellum (see Grahn, 2009, and Large, 2010, for related reviews). The basal ganglia (i.e., a collection of structures residing beneath the cortex in the midbrain) may play a special role in the internal generation of the beat (Grahn & Brett, 2007). The cerebellum (i.e., the corrugated structure located at the lower backend of the brain) Empirical Musicology Review Vol. 7, No. 1-2, 2012 19 is thought to house ‘internal models’ that can be used to simulate actions—including their temporal properties—before they are carried out (Ito, 2008; Wolpert, Miall, & Kawato, 1998) and, in this sense, it may contribute to expectancy generation. Rhythmic grouping recruits brain regions implicated in auditory attention and working memory (Chapin, Zanto, Jantzen, Kelso, Steinberg, & Large, 2010), for example, the dorsolateral and ventrolateral prefrontal cortices (Chen, Penhune, & Zatorre, 2008). In addition to the above brain regions, the auditory cortices participate in grouping and expectancy generation in the context of music and speech (Rauschecker, 2011). The forgoing suggests that a duality of meter and grouping is evident (1) in formal models of temporal perception, (2) in functional mechanisms related to expectancies, and (3) in the brain bases of these processes and mechanisms. Although London (2012) does not specifically address the implications of the meter/grouping duality for linguistics, the relationship between temporal perception, expectancies, and the brain circuits that process hierarchical and grouping structure in linguistic prosody and music is a topic of growing interest (Patel, 2008). The promising nature of this topic is highlighted by the fact that neuroscientific research on prosodic cues to syntactic structure and emotion (e.g., Aziz-Zadeh, Sheng, & Gheytanchi, 2010; Strelnikov, Vorobyev, Chernigovskya, & Medvedev, 2006), as well as work on links between perception and action in speech and music (e.g., Rauschecker, 2011), have identified similar brain regions to those associated with meter and grouping in musical rhythm. PERCEPTION AS COVERT ACTION London (2012) points out that rhythm perception is an active rather than passive process. This can be understood in at least two ways. First, the listener does not merely abstract information from acoustical features of a rhythmic pattern in a ‘bottom-up’ fashion, but also bestows structure on the pattern via ‘top-down’ processes, such as attentional resource allocation and mental schemas (i.e., internalized knowledge structures that bias perceptual processing and cognitive interpretation). London (2012) gives the example of subjective rhythmization, which he prefers to call subjective metricization because the phenomenon involves imagining accents that imply the grouping of identical, isochronous tones into metric units. Rhythm perception is active in another sense: It engages the listener’s body and its potential for movement (Phillips-Silver, 2010; Shove & Repp, 1995). Thus, auditory rhythm is a powerful means of inducing movement, and the mechanisms underlying such induction may be causally related to the perceptual experience of rhythmic phenomena including beat, meter, and grouping. The relation between movement and rhythm appears to be a fairly direct one. The basic periodicities of beat and meter are lawfully related to various periodic body movements, such as walking (Styns, van Noorden, Moelants, & Leman, 2007), and metric hierarchies are embodied in people’s dance moves (Leman & Naveda, 2010; Toiviainen, Luck, & Thompson, 2010). The correspondence is not limited to periodicity per se, however, but also extends to changes in period. A well-documented case concerns the proposal that tempo changes have similar trajectories to human movement. For example, the final ritard (a slowing of tempo at the end of a piece) is timed similarly to the footfalls of a runner coming to a halt (Friberg & Sundberg, 1999). It therefore seems reasonable to assume that listening to such a tempo change may evoke experiences related to such movement in the listener. Some researchers have gone one step further by arguing, more generally, that when listening to music we hear communicative gestures of a virtual social agent with whom the listener can interact in a meaningful and affective manner (Leman, 2007; see also Godøy & Leman, 2010). Thus, individuals understand music in the same way that they understand others’ intentions during social interaction. More specifically, expressive intentions are attributed to music because patterns of sonic energy evoke bodily gestures that the listener can interpret meaningfully based on his or her personal experience of interacting with others in the world. This fits with embodied cognition views (e.g., Lakoff & Johnson, 1999; Varela, Thompson, & Rosch, 1992), which hold that the way in which we perceive and think about objects and events in the world is determined by how our bodies can interact with them. The notion of embodiment is formalized in a class of theories that postulate the involvement of the motor and vestibular systems, in addition to the auditory sensory system, in musical rhythm perception (e.g., Todd, Lee, & O’Boyle, 2002; Todd, O’Boyle, & Lee, 1999). The main thrust of these theories is that movement-related processes in the brain resonate in sympathy with the temporal structure of rhythmic auditory input (van Noorden & Moelants, 1999). Resonances can occur at multiple levels that correspond to the metric structure of musical rhythm. Evidence in support of sensorimotor theories of rhythm is steadily mounting. Relevant studies have shown that being moved in time with rhythmic stimuli can bias the perception of meter (Phillips-Silver & Trainor, 2005), and that a listener’s body proportions affect preferences for musical tempo (e.g., broad shouldered individuals prefer relatively slow tempi; Todd, Cousins, & Lee, 2007). Empirical Musicology Review Vol. 7, No. 1-2, 2012 20 Another source of evidence is found in neuroscientific research. As London mentions in his section on meter versus grouping, the entrainment that underlies the experience of musical meter relies on links between auditory and motor regions of the brain. Work employing fMRI has shown that rhythm perception recruits brain areas that are traditionally considered to be part of the motor system. Thus, areas involved in motor control and learning, action selection, planning, and sequencing are activated in tasks that require beat finding, imagined sensorimotor synchronization, and recognition memory for rhythm patterns (Grahn, 2009; Janata & Grafton, 2003; Oullier, Jantzen, Steinberg, & Kelso, 2005; Zatorre, Chen, & Penhune, 2007). One possibility, then, is that rhythm perception involves the simulation of action. Action simulation occurs when sensorimotor neural processes that resemble those associated with executing an action are engaged in the absence of overt movement (see Decety & Grezes, 2006; Gallese, Keysers, & Rizzolatti, 2004; Wilson & Knoblich, 2005) or, more precisely, are not the immediate cause of such movement. One way to conceive simulation is as imagined action (i.e., where ‘action’ refers to movements plus their effects). This process does not only occur offline (as when imagining a past or future action) but also online during action observation. It has been proposed that online action simulation facilitates the understanding of others’ intentions and affective states, as well as playing a role in predicting an observed action’s immediate outcome and future course (e.g., Sebanz & Knoblich, 2009; Schütz-Bosbach & Prinz, 2007; Wilson & Knoblich, 2005). Recent work has also linked action simulation to the aesthetic appreciation of art forms including dance, music, and even pictures and sculpture (Cross & Ticini, 2011; Freedberg & Gallese, 2007; Molnar-Szakacs & Overy, 2006). As action simulation is mediated by associations between sensory and motor processes, it is modulated by the observer’s own action experience (Bangert et al., 2006; Baumann, Koeneke, Meyer, Lutz, & Jäncke, 2005; Haueisen & Knösche, 2001). Basically, simulation is strong to the extent that experience is high. Auditory-motor simulation is, therefore, especially strong in musicians due to their experiences at playing and observing others play music (Lahav, Saltzmann, & Schlaug, 2007). This implies that there may be individual differences in the degree to which rhythm perception is active. We know that musical experience shapes the brain—quite literally—in specific ways. Work with anatomical MRI (which can be used to examine features such as cortical thickness and white matter fibers consisting of axons that connect neurons) has revealed brain structural changes indicating plasticity related to musical training. For example, cortical regions dealing with pitch perception are enlarged in musicians relative to non-musicians, and instrument-specific patterns of plasticity, such as enlarged cortical representation of the fingers of the left hand in violinists, can also be observed (Bangert & Schlaug, 2006; Elbert, Pantev, Wienbruch, Rockstroh, & Taub, 1995; Schlaug, 2001). Most germane, however, is recent evidence (Halwani, Loui, Rüber, & Schlaug, 2011) that musical training affects the connectivity of auditory and motor areas that are implicated in rhythm perception and the simulation of musical action. It may be the case that music-specific adaptations of tracts connecting these regions contribute to the optimal processing of musical rhythms by influencing the strength of perception-action coupling. Further evidence that rhythm perception may depend on anatomical connectivity between auditory and motor brain regions comes from studies of the rhythmic behavior of non-human animals. In this vein, Ani Patel and others have claimed that only vocal learners (e.g., various birds, including cockatoos) are capable of synchronizing their movements with an external beat (Patel, Iversen, Bregman, & Schulz, 2009). The source of an animal’s motivation to do so is another matter. In any case, studies of vocal learning in songbirds have highlighted the importance of auditory-motor connectivity (see Bolhuis, Okanoya, & Scharff, 2010). The link with vocal learning logically points to language. It is interesting to note in this regard that ‘motor theories’ of speech perception have a venerable history (e.g., Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967; Liberman & Mattingly, 1985). Recently, the power of perception-action coupling has been demonstrated by the finding that active rhythmic components of therapy for aphasics aid fluent speech (Stahl, Kotz, Henseler, Turner, & Geyer, 2011). MULTIPLE TIME SCALES London (2012) points out that processes related to music perception and production are yoked differentially to the various time scales that characterize music’s hierarchical structure. These time scales are centered on a zone of maximal pulse salience that spans the 400-900 ms range (Parncutt, 1994). London’s observation that different levels in musical structure “play by different rules” (London, 2012, p. 7) reflects that fact that different levels favor different processing operations. The reason behind this could be that they recruit different neural processes. Neurophysiological work with animal models has yielded evidence that distinct neural timing circuits favor different time scales (Buhusi & Meck, 2005). Although generalizing this work to humans Empirical Musicology Review Vol. 7, No. 1-2, 2012 21 is risky, there are some reasonable hypotheses that can be advanced. One that is currently in circulation (see Schwartze, Keller, Patel, & Kotz, 2011) is that the timing functions of the cerebellum are specialized for the pre-attentive processing of information with fine temporal resolution, while corticostriato-thalamo-cortical loops, which comprise structures in the midbrain and (pre)frontal cortex, handle attention-dependent processing of information at timescales that could serve as a musical tactus or supertactus. Research with clinical populations with brain lesions in the cerebellum and basal ganglia is broadly consistent with such a distinction (e.g., Ivry, 1996; Schwartze et al., 2011), though the exact role of these structures in temporal processing is far from settled (see Diedrichsen, Ivry, & Pressing, 2003). Lesions may disrupt the communication of neurons in processing loops spanning segregated brain regions, potentially affecting a wide variety of processes that rely on synchronized neural activity. There is empirical evidence that the degree of synchrony in brain activity at multiple levels in large-scale networks may be indicative of different sensory and motor processes (see Neuper & Klimesch, 2006; Sporns, 2011). Relevant work typically involves examining frequency bands in electric or magnetic fields generated by neuronal activity, as measured by EEG or MEG. It has been revealed that synchronized (as well as desynchronized) neural oscillations occur in different bands of the EEG/MEG frequency spectra: gamma (~30-100 Hz [cycles per second]), beta (13-30 Hz), alpha (713 Hz), theta (4-7 Hz), and delta (1-4 Hz). Synchronous oscillations can also be elicited at longer periods (e.g., as seen in steady-state evoked potentials to periodic stimuli), and spontaneous long-range correlations and fractal scaling (e.g., 1/f noise) can be observed in brain activity (e.g., Werner, 2010). The physiological mechanisms reflected in distributed neural oscillations in these different frequency bands purportedly serve different psychological functions, and these may be linked to different aspects of musical rhythm perception (see Osborne, 2009). Thus, limits in the amount of information that can be processed simultaneously at different levels of hierarchical musical structure (Swain, 1986) may be constrained by the dynamics of neural oscillations at multiple time scales. Processes at levels that are low and fast (in the gamma to delta range) when it comes to musical structure have been linked to a number of functions, including the perceptual binding of features of objects or events, the matching of perceptual information with the contents of memory, and sensorymotor integration (e.g., Herrmann, Munk, & Engel, 2004; Ribary, 2005). Transient activity at these timescales provides objective markers of temporal expectancy and top-down influences (i.e., subjective accents) on the metric interpretation of rhythmic stimuli with periodicities at longer timescales (e.g., Iversen et al., 2009; Zanto et al., 2006). Related work has revealed oscillatory activity (reflected in steady-state evoked potentials) at timescales that are relevant to the musical tactus (Nozaradan et al., 2011), which suggests the operation of mechanisms that are capable of supporting entrainment and perception-action coupling in musical contexts. At higher, slower levels (2-3 second periods)—corresponding to musical measures or phrases—neural oscillations may play a role in integrating, or ‘chunking’, multiple events that are extended in time into unitary Gestalts, as well as contributing to the sense of the psychological present (Pöppel, 2004). This latter concept is particularly germane to London’s (2011) treatment of time scales in musical rhythm. The psychological, or subjective present is “characterized by the phenomenal impression of ‘nowness’ ” (Pöppel, 2004, p. 300). Many estimates of the duration of the psychological present place it at around 3 s, but some extend it to the 5-6 s range mentioned by London (2012) (see Fraisse, 1957). The psychological present of course applies generally to how humans experience their temporal passage through the world. Its relevance therefore extends beyond music perception, and an auspicious link to language has been made: Three seconds corresponds to the average line length in metered poetry from diverse cultural traditions (Turner & Poeppel, 1983). Although metered poetry is a domain where musical rhythm and speech rhythm may be close cousins, the place of natural speech is less clear in this regard. London (2012) acknowledges this by treading carefully into the minefield of isochrony in speech. Some have championed the notion that speech is timed in accordance with underlying periodicities (e.g., Ladefoged, 1975; Pike, 1945). The ease with which speech can be produced with periodic accents (e.g., when speaking in time with a metronome) is certainly suggestive of underlying attractor dynamics based on oscillatory processes (e.g., Cummins & Port, 1998; Port, 2003). Nevertheless, the notion of isochrony in everyday conversational speech is not a view that is widely subscribed to (see Patel, 2008). This does not mean that musical and speech rhythm are completely independent, however, for a number of reasons. First, it can be noted that the pulse in human-produced music typically deviates quite obviously from isochrony, in large part due to variations in local tempo that performers introduce for expressive purposes (Gabrielsson, 2003; Palmer, 1997). Still, it is true, as London (2012) notes, that quasi-periodicities generally tend to be sustained for longer amounts of time in music than in speech. Second, metric structure in music is not necessarily isochronous at all levels, as London (2012) observes. In fact, the music of many (predominantly non-Western) cultures contains nested nonEmpirical Musicology Review Vol. 7, No. 1-2, 2012 22 isochronous levels of pulsation to accommodate the ‘uneven’ ratios that flavor their rhythms (Hannon & Trehub, 2005; London, 2004). Moreover, it is questionable whether overlap in speech/musical rhythms should be sought in comparisons between natural speech and rehearsed performances of premeditated musical compositions, or, more futile still, rhythms as notated symbolically in a musical score: The rhythms of the ‘free’ musical improvisation—wherein performers often eschew a sense of pulse—may be more appropriate. Finally, it worth noting that, although the concept of isochrony is controversial in speech, the notion of hierarchical structure is not: Syntax and grammar entail just this. Indeed, the hierarchical structuring of linguistic grammar—and, arguably, grammars underlying many forms of skilled action (see Fadiga, Craighero, & D'Ausilio, 2009; Novembre & Keller, 2011)—may serve functions related to those fulfilled by metric hierarchies in musical rhythm (e.g., as schemes for expectancy generation). This may be the case especially in naturalistic settings where speech is accompanied by physical body gestures that are continuous and also evolve at multiple time scales. Thus, a hypothesis that is worthy of attention is that, while speech and musical rhythm may differ markedly when it comes to the auditory signal, the rhythm of body movements accompanying both activities may in fact be similar in terms of structure and function.